Adaptive Semantic-Spatio-Temporal Graph Convolutional Network for Lip Reading

The goal of this work is to recognize words, phrases, and sentences being spoken by a talking face without given the audio. Current deep learning approaches for lip reading focus on exploring appearance optical flow information videos. However, these methods do not fully exploit characteristics motion. In addition flow, mouth contour deformation usually conveys significant that complementary others. modeling dynamic has received little attention than flow. work, we propose novel model contours called Adaptive Semantic-Spatio-Temporal Graph Convolution Network (ASST-GCN), go beyond previous automatically both spatial temporal from To combine contour, two-stream visual front-end network proposed. Experimental results demonstrate proposed method significantly outperforms state-of-the-art several large-scale benchmarks.